feat(hackathon): add unitree-go2-hackathon blueprint by publu · Pull Request #2301 · dimensionalOS/dimos

publu · 2026-05-28T17:21:01Z

Summary

Adds the unitree-go2-hackathon blueprint — a self-contained agentic stack for the Unitree Go2
PerceptionLoopModule: shared YOLO + MobileCLIP at 15 fps, SceneBuffer with 2s TTL, process-singleton model caching
SmartFollowSkillContainer: follow person (CLIP-based ID) or object (YOLO class) with SpinSearch auto-recovery
FindSkillContainer: autonomous explore + camera watch + annotate-when-found (stops exploration on match)
DogModeModule: threat-based bark/growl state machine (WANDERING → ALERT → CLOSE), afplay sound effects
frame_writer: debug overlay utilities for annotated MJPEG frames
All modules are fully contained under dimos/robot/unitree/go2/blueprints/hackathon/

Test plan

python -c "from dimos.robot.unitree.go2.blueprints.hackathon.blueprint import unitree_go2_hackathon" imports cleanly
Start stack with -b unitree-go2-hackathon — PerceptionLoop starts, MCP server exposes smart_follow_person, smart_follow_object, smart_find, start_dog_mode
daneel_smart_follow_object("chair") locks on to a visible chair without spinning
daneel_start_dog_mode() barks at approaching people

🤖 Generated with Claude Code

greptile-apps · 2026-05-28T17:24:18Z

Greptile Summary

This PR introduces a self-contained hackathon blueprint for the Unitree Go2, adding five new modules under dimos/robot/unitree/go2/blueprints/hackathon/: a shared YOLO+MobileCLIP perception loop, a person/object follower with SpinSearch recovery, an explore-and-find skill, an autonomous dog-behavior state machine, and a debug frame-writer for dashboard overlays.

PerceptionLoopModule acts as a process-singleton L2 cache running YOLO at ~15 fps with MobileCLIP embeddings in a SceneBuffer; all other modules consume it via RPC to avoid duplicate model loads (~1.85 GB saved per skill).
SmartFollowSkillContainer and FindSkillContainer implement the follow and search skills, but smart_follow_object incorrectly registers its async tool callback under the "smart_follow_person" name, so MCP clients never receive a terminal result when that skill is invoked.
DogModeModule runs a threat-based bark/growl state machine but blocks the main control thread for 3.2 s during sniff animations without checking the stop event, making graceful shutdown slow.

Confidence Score: 3/5

Functional for demos but two present defects need fixing before reliable operation: smart_follow_object never delivers its result to the MCP client, and the follow/approach threads can die silently on any image gap.

The wrong tool-name registration in _start_follow means smart_follow_object never completes from the agent's perspective — a real behavioral defect that would show up immediately during testing. The assert image is not None pattern in both _follow_loop and _approach_loop means any transient image-stream hiccup kills the control thread silently, leaving the robot in an unknown state. The 3.2 s blocking sleep in the dog-mode sniff path makes graceful shutdown sluggish and less predictable.

smart_follow.py (wrong tool name + assert guard) and dog_mode.py (blocking sleep in stop path) need the most attention before this is used in a real demo.

Important Files Changed

Filename	Overview
dimos/robot/unitree/go2/blueprints/hackathon/blueprint.py	Wires the hackathon agentic stack together using autoconnect; straightforward composition with no logic issues.
dimos/robot/unitree/go2/blueprints/hackathon/smart_follow.py	Two bugs: `start_tool("smart_follow_person")` is always used even for the object-follow path (wrong tool name), and `assert image is not None` silently kills the control thread on any image-stream gap.
dimos/robot/unitree/go2/blueprints/hackathon/dog_mode.py	State machine logic is sound, but `time.sleep(3.2)` in the hot loop ignores `_should_stop`, delaying graceful shutdown; unused `subprocess` import left in.
dimos/robot/unitree/go2/blueprints/hackathon/find.py	Find + explore logic looks correct; minor: hardcoded `img_w=1280` fallback always fires (the `_Det` wrapper never has an `.image` attribute), and the `timeout_s` docstring claims "default 120s" when the actual default is unbounded.
dimos/robot/unitree/go2/blueprints/hackathon/perception_loop.py	Process-singleton YOLO+CLIP loop is well-designed; the only nit is a dead `if True:` block that should be cleaned up.
dimos/robot/unitree/go2/blueprints/hackathon/frame_writer.py	Dashboard overlay utilities — writes annotated JPEG and JSON state to /tmp; no issues found.

Sequence Diagram

sequenceDiagram
    participant MCP as MCP Client
    participant SF as SmartFollowSkillContainer
    participant PL as PerceptionLoopModule
    participant VS as VisualServoing2D
    participant ROS as cmd_vel (Out)

    MCP->>SF: smart_follow_object("chair")
    SF->>PL: get_shared_buffer() [SceneBuffer 2s TTL]
    PL-->>SF: "{track_id: 5, name: "chair", bbox: [...]}"
    SF->>SF: start_tool("smart_follow_person") ⚠️ wrong name
    SF->>SF: spawn _follow_loop thread

    loop Every 1/15s
        SF->>PL: "current_detections(class_filter="chair")"
        PL-->>SF: "[{track_id, bbox, name, confidence}]"
        SF->>VS: compute_twist(target.bbox, image.width)
        VS-->>SF: Twist
        SF->>ROS: publish(Twist)
    end

    SF->>MCP: tool_update("smart_follow_person", "Following stopped.") ⚠️ wrong name
    SF->>MCP: stop_tool("smart_follow_person") ⚠️ MCP never resolves smart_follow_object

Comments Outside Diff (2)

dimos/robot/unitree/go2/blueprints/hackathon/smart_follow.py, line 1453 (link)

Wrong tool name for object-follow path

_start_follow always calls self.start_tool("smart_follow_person") regardless of mode, so when smart_follow_object invokes _start_follow("chair", mode="object"), the async tool result is registered under "smart_follow_person". Every tool_update and stop_tool call in _follow_loop also references "smart_follow_person", so the agent callback for smart_follow_object never fires — the MCP client will never receive a terminal result for that invocation.
dimos/robot/unitree/go2/blueprints/hackathon/smart_follow.py, line 1497 (link)

assert silently kills control thread on image-stream gap

assert image is not None is the only guard against a None image inside _follow_loop. If the image subscription briefly stalls (e.g., transient WebRTC gap), the assertion raises AssertionError, which propagates uncaught in the daemon thread, killing the follow loop while _should_stop remains unset and self._thread still holds the dead reference. The robot silently stops reacting without reporting an error to the MCP client. The same pattern appears in _approach_loop at the corresponding location. A simple if image is None: time.sleep(period); continue is the safe alternative.

_{Reviews (1): Last reviewed commit: "feat(hackathon): add sounds/perry-the-pl..." | Re-trigger Greptile}

greptile-apps · 2026-05-28T17:24:28Z

+                            "bbox": p.bbox,
+                            "track_id": tid,
+                            "threat": self._threats.get(tid),
+                        })


Blocking sleep in main control loop ignores stop signal

time.sleep(3.2) blocks _dog_loop for the full sniff animation duration without checking _should_stop. If stop_dog_mode() is called during a sniff, _thread.join(timeout=DEFAULT_THREAD_JOIN_TIMEOUT) in the stop path will block for at least 3.2 s before the thread can check _should_stop.is_set(). Replacing this with a short-poll (self._should_stop.wait(timeout=3.2)) lets the loop exit immediately on stop.

greptile-apps · 2026-05-28T17:24:29Z

+            # Frame annotation — every loop iteration (was throttled to every
+            # 3rd, which capped the dashboard camera at ~5fps; YOLO is the real
+            # gate so writing every frame just keeps up with the loop).
+            if True:


if True: is dead code left over from a previous throttle-removal refactor. It adds no branching and confuses readers about intent.

Suggested change

# Frame annotation — every loop iteration (was throttled to every

# 3rd, which capped the dashboard camera at ~5fps; YOLO is the real

# gate so writing every frame just keeps up with the loop).

if True:

# Frame annotation — every loop iteration (YOLO is the rate gate).

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

greptile-apps · 2026-05-28T17:24:34Z

+import random
+import subprocess
+import time


subprocess is imported but never used in this module — sound playback was moved to sounddevice/soundfile. Leaving it in triggers linting warnings and misleads readers.

Suggested change

import random

import subprocess

import time

import random

import time

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

greptile-apps · 2026-05-28T17:24:36Z

+            query: What to look for, e.g. "red chair", "dog", "person with backpack"
+            explore: If True (default), start autonomous exploration while searching.
+            then_approach: If True, automatically approach and get close after finding.
+            timeout_s: Give up after this many seconds (default 120s).


The docstring states timeout_s defaults to 120 s, but the parameter default is 0.0 and the code treats <= 0 as unbounded. As written, smart_find runs forever by default, which contradicts the doc.

Suggested change

timeout_s: Give up after this many seconds (default 120s).

timeout_s: Give up after this many seconds (0 = no timeout, runs until found or stopped).

greptile-apps · 2026-05-28T17:31:50Z

Greptile encountered an error while reviewing this PR. Please reach out to support@greptile.com for assistance.

publu · 2026-05-28T17:39:38Z

@greptile review

publu added 11 commits May 29, 2026 01:19

feat(hackathon): add __init__.py

455763b

feat(hackathon): add blueprint.py

3c8801d

feat(hackathon): add frame_writer.py

4926799

feat(hackathon): add perception_loop.py

0ea99be

feat(hackathon): add smart_follow.py

1d5c72a

feat(hackathon): add find.py

5c033c0

feat(hackathon): add dog_mode.py

0ae84a3

feat(hackathon): add sounds/deepbark.mp3

4175716

feat(hackathon): add sounds/dog-growling-and-barking.mp3

0331b67

feat(hackathon): add sounds/minecraft-dog-bark.mp3

1c92fd8

feat(hackathon): add sounds/perry-the-platypuss-growl.mp3

1b2bb3e

publu requested a review from a team May 28, 2026 17:21

publu requested review from leshy, mustafab0, paul-nechifor and spomichter as code owners May 28, 2026 17:21

greptile-apps Bot reviewed May 28, 2026

View reviewed changes

publu added 4 commits May 29, 2026 01:29

fix: address greptile review — smart_follow.py

0d926e7

fix: address greptile review — dog_mode.py

0fb5302

fix: address greptile review — perception_loop.py

3837179

fix: address greptile review — find.py

43b4876

leshy added the hackaton label May 29, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(hackathon): add unitree-go2-hackathon blueprint#2301

feat(hackathon): add unitree-go2-hackathon blueprint#2301
publu wants to merge 15 commits into
dimensionalOS:mainfrom
publu:feat/hackathon-blueprint

publu commented May 28, 2026

Uh oh!

greptile-apps Bot commented May 28, 2026 •

edited

Loading

Comments Outside Diff (2)

Uh oh!

greptile-apps Bot May 28, 2026

Uh oh!

greptile-apps Bot May 28, 2026

Uh oh!

greptile-apps Bot May 28, 2026

Uh oh!

greptile-apps Bot May 28, 2026

Uh oh!

greptile-apps Bot commented May 28, 2026

Uh oh!

publu commented May 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	timeout_s: Give up after this many seconds (default 120s).
	timeout_s: Give up after this many seconds (0 = no timeout, runs until found or stopped).

Conversation

publu commented May 28, 2026

Summary

Test plan

Uh oh!

greptile-apps Bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 3/5

Important Files Changed

Sequence Diagram

Comments Outside Diff (2)

Uh oh!

greptile-apps Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented May 28, 2026

Uh oh!

publu commented May 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

greptile-apps Bot commented May 28, 2026 •

edited

Loading